Skip to content

test: materialize deterministic MCP corruption fixtures#162

Merged
ProfRandom92 merged 4 commits into
mainfrom
codex/add-deterministic-mcp-corruption-fixture-materialization
May 21, 2026
Merged

test: materialize deterministic MCP corruption fixtures#162
ProfRandom92 merged 4 commits into
mainfrom
codex/add-deterministic-mcp-corruption-fixture-materialization

Conversation

@ProfRandom92

Copy link
Copy Markdown
Owner

Summary:

  • Add a deterministic materializer that reads artifacts/mcp_trace_corruption_manifest.json and materializes corruption fixtures for exactly three operators: DROP_APPROVAL_GATE, REMOVE_DEPENDENCY_EDGE, and TRUNCATE_RECOVERY_PATH while preserving the original fixture schema (trace.json, dependency_graph.json, state.json).
  • Materialize the committed corpus under fixtures/mcp_trace_replay_corruptions/<source_fixture_slug>/<operator_slug>/ for the three existing MCP families (degraded, mild, moderate).
  • Add focused tests that assert directory presence, operator scope, file-shape preservation, single-surface mutations, deterministic regeneration, absence of environment/time tokens, and manifest/failure-label alignment, and leave artifacts/evidence_index.json unchanged.

Changed files:

  • scripts/materialize_mcp_trace_corruptions.py (new deterministic materializer).
  • tests/test_mcp_trace_corruption_materialization.py (new focused materialization tests).
  • fixtures/mcp_trace_replay_corruptions/** (new materialized fixture directories for the three selected operators across three source families).

Testing:

  • Ran python scripts/materialize_mcp_trace_corruptions.py successfully to produce fixtures/mcp_trace_replay_corruptions/.
  • Ran pytest -q tests/test_mcp_trace_corruption_materialization.py which passed (8 passed).
  • Ran npm run check which completed successfully with full test-suite green (299 passed).

Risks:

  • Low risk because changes are limited to fixture generation and tests and do not modify validators, workflows, package files, or failure labels.
  • Materializer enforces manifest applicability and will raise deterministic repo-relative errors if source fixtures are missing or malformed.

Next:

  • Materialize remaining operators (SWAP_TOOL_ORDER, INSERT_UNVALIDATED_ACTION, COLLAPSE_CAPABILITY_BOUNDARY) in follow-up PRs and optionally wire these fixtures into validator-level checks.

Codex Task

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a materialization script and a corresponding test suite to generate deterministic MCP trace corruption fixtures from a manifest, alongside a comprehensive set of JSON fixtures for various failure scenarios. The reviewer feedback consistently identifies a violation of repository rules regarding JSON processing: null values for expected list fields should be treated as empty lists rather than immediately raising a RuntimeError. Actionable code suggestions were provided for several functions to ensure robust handling of missing or null list data in the manifest, state, graph, and trace files.

Comment thread scripts/materialize_mcp_trace_corruptions.py Outdated
Comment thread scripts/materialize_mcp_trace_corruptions.py Outdated
Comment thread scripts/materialize_mcp_trace_corruptions.py Outdated
Comment thread scripts/materialize_mcp_trace_corruptions.py Outdated
@ProfRandom92 ProfRandom92 merged commit 5ebda3d into main May 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant